Skip to content

Time range components - #394

Merged
thorrester merged 6 commits into
mainfrom
time-range-components
Apr 29, 2026
Merged

thorrester merged 6 commits into
mainfrom
time-range-components

Conversation

@thorrester

@thorrester thorrester commented Apr 28, 2026 •

Copy link
Copy Markdown
Member

Pull Request

Short Summary

Adds a faceted filter system to the trace dashboard — service, status code, duration, has-errors, and span attributes — plus a reusable DashboardTimeBar component that consolidates the time-range picker and refresh button. Also extends the Rust Scouter proxy with three new trace endpoints, a full GenAI metrics route module, and an observability metrics route.

Context

Trace filter system

The trace dashboard previously had a free-text search box and no way to filter by service, status code, or duration without knowing the exact query syntax. This PR replaces that with a proper faceted filter layer:

  • filterState.svelte.ts — pure derive function that maps TracePageFilter to an ActiveFilter[] chip list, plus a removeActiveFilter helper. No reactive class, no global store — just functions the dashboard calls.
  • FacetSidebar.svelte — persistent sidebar listing service names and status codes with counts from the backend, plus duration range and attribute inputs.
  • ChipBar.svelte + AddFilterMenu.svelte — chip strip showing active filters with individual X buttons; the + button opens AddFilterMenu which lets you pick service, status, has-errors, attribute key=value, or a duration range.
  • AttributeFacet, DurationFacet, FacetRow, FacetSection — composable primitives used by both the sidebar and the add-filter menu.

The dashboard fetches TraceFacetsResponse (service counts, status code counts) alongside metrics and page data on every time-range change or manual refresh. Facet data feeds both the sidebar and the chip bar.

DashboardTimeBar component

The inline <TimeRangeFilter> + <button>Refresh</button> pair was duplicated across dashboards. DashboardTimeBar.svelte wraps both with a consistent layout and exposes onRangeChange / onRefresh props. TraceDashboard and the agent observability page now use it. timeState.svelte.ts gains isRefreshing state with beginRefresh() / endRefresh() guards to prevent overlapping fetches.

ModeTabs — search vs analytics

The dashboard now has two modes: search (trace list + filters) and analytics (metric charts). A ModeTabs component switches between them. Previously, charts and table were always both rendered.

Rust: new Scouter proxy endpoints

scouter/trace/route.rs adds three routes:

Endpoint What it does
POST /scouter/trace/spans/filters Fetch spans matching a TraceFilters body
POST /scouter/trace/facets Return service/status facet counts for a time range
GET /scouter/trace/{id}/spans Fetch spans for a specific trace ID

scouter/genai/route.rs is a new ~526-line module proxying all GenAI Scouter routes (token metrics, operation breakdown, model usage, agent activity, tool activity, error breakdown, spans, conversations). scouter/observability/route.rs proxies the observability metrics endpoint.

ScouterApiClient in client.rs was refactored — execute_request and build_route_url extracted from the inline match arms, and request_with_path added for routes that include dynamic path segments (e.g. trace/{id}/spans).

Existing trace endpoints now attach AuditContext extensions — they were missing them before.

Python / Scouter SDK

  • service_map/middleware.py — new service-map middleware for automatic trace enrichment.
  • tracing/middleware.py — tracing middleware refactored into its own file.
  • bifrost/ — split from bifrost.py into a package (_client.py + __init__.py).
  • evaluate/runner.py — eval runner gets concurrency improvements.
  • Agent examples under py-opsml/examples/genai/agent/ are replaced: music_recommender, recipe_generator, and single_agent (bespoke, divergent setups) are removed; crewai/, google/, and openai/ examples replace them with a shared setup.py / scenarios.jsonl / run.py harness.

Test additions

Vitest unit tests cover filterState, DashboardTimeBar, agent assertion helpers, dashboard/utils, a2aOperations, agentInference, trace utils, and trace facets. Rust integration tests cover the new GenAI, observability, trace, and profile endpoints.

File Change
trace/filters/filterState.svelte.ts New — derive active-filter chips from TracePageFilter
trace/filters/FacetSidebar.svelte New — facet sidebar with service/status/duration/attribute sections
trace/filters/ChipBar.svelte New — active filter chip strip + add-filter button
trace/filters/AddFilterMenu.svelte New — dropdown menu for adding filters
utils/DashboardTimeBar.svelte New — shared time-range picker + refresh button wrapper
utils/timeState.svelte.ts Adds isRefreshing guard with beginRefresh/endRefresh
trace/TraceDashboard.svelte Wires facets, ChipBar, FacetSidebar, ModeTabs; uses DashboardTimeBar
trace/TraceTable.svelte Removes MultiComboSearchBox; clears trace_id URL param on sidebar close
scouter/trace/route.rs Adds 3 new trace endpoints; attaches AuditContext to existing ones
scouter/genai/route.rs New — full GenAI metrics proxy (~526 lines)
scouter/observability/route.rs New — observability metrics proxy
scouter/client.rs Extracts execute_request/build_route_url; adds request_with_path
py-opsml/python/opsml/scouter/service_map/ New — service-map middleware package
py-opsml/python/opsml/scouter/tracing/middleware.py New — tracing middleware in own file
py-opsml/examples/genai/agent/ Replaces 3 bespoke examples with crewai/google/openai + shared harness

Is this a Breaking Change?

No. All new Svelte components are additive; TraceDashboard and TraceTable props retain backward-compatible defaults. New Rust endpoints are additive. The ScouterApiClient refactor preserves the existing request method signature. Python example deletions remove content not imported by library consumers.

@github-actions

github-actions Bot commented Apr 28, 2026 •

Copy link
Copy Markdown

Dependency Review

✅ No vulnerabilities or license issues found.

Scanned Manifest Files

@thorrester
thorrester marked this pull request as ready for review April 28, 2026 02:04
devin-ai-integration[bot]

This comment was marked as resolved.

thorrester and others added 2 commits April 28, 2026 08:29
…/card/[space]/[name]/[version]/evaluation/+page.ts

Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 new potential issue.

View 10 additional findings in Devin Review.

Open in Devin Review

Comment on lines 306 to +314
const path = promptEvalPath(e);
const profile = e.monitoringData.profile as AgentEvalProfile;
e.monitoringData.selectedData.workflows?.items?.forEach(w =>
items.push({ ...w, _agentName: e.promptCard.name, _evalPath: path, _profile: profile })
items.push({
...w,
_agentName: e.promptCard.name,
_evalPath: path,
_profile: profile,
_traceId: recordTraceMap.get(w.record_uid),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 _traceId lookup fails when records and workflows are on different pages

The workflowPage derived block builds a recordTraceMap from the current page of records (e.monitoringData.selectedData.records?.items), then sets _traceId: recordTraceMap.get(w.record_uid) for each workflow. Since records and workflows are paginated independently via handleRecordPageChange and handleWorkflowPageChange (crates/opsml_server/opsml_ui/src/lib/components/card/agent/evaluation/AgentEvalDashboard.svelte:98-122), after the user pages forward on either table, the map only contains UIDs from the current record page. Any workflow referencing a record on a different page will have _traceId set to undefined, silently hiding the trace link even when the record actually has a trace_id.

(Refers to lines 293-314)

Prompt for agents
The _traceId for each workflow is looked up from a map built from only the currently-visible page of records. Since records and workflows paginate independently (handleRecordPageChange vs handleWorkflowPageChange), a workflow on page N may reference a record_uid that is on a different records page, causing the map lookup to return undefined and the trace link to disappear.

Possible approaches:
1. Maintain a cumulative record trace map (Map<string, string>) across all pages ever loaded, not just the current page. Append to it in the effect whenever new records arrive.
2. Include trace_id directly on the AgentEvalWorkflowResult type from the backend, so workflows carry their own trace_id without needing to cross-reference records.
3. If the relationship is always 1:1, denormalize by having the backend embed trace_id in the workflow response.

Files involved: AgentEvalDashboard.svelte (workflowPage derived block), types.ts (WorkflowWithAgent), and potentially the backend workflow endpoint.
Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

@thorrester
thorrester changed the base branch from bump-scouter to main April 29, 2026 11:52
@thorrester
thorrester merged commit 40db314 into main Apr 29, 2026
24 of 25 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant